Enriching geometric digital twins of buildings with small objects by fusing laser scanning and AI-based image recognition

This paper addresses the challenge of enriching geometric digital twins buildings, with a particular emphasis on capturing small but important entities from electrical and fire-safety domain, such as signs, sockets, switches, smoke alarms, etc. Unlike most previous research that focussed structural elements processed laser point clouds images separately, we propose novel method fuses scanning photogrammetry methods to capture relevant objects, recognise them in 2D then map these 3D space. The considered object classes include (light switch, light, speaker, socket, elevator button), safety (emergency alarm, fire extinguisher, escape sign), plumbing system (pipes), other objects useful information (door sign, board). Semantic like class labels is extracted by applying AI-based image segmentation mapped cloud, segmenting cloud into clusters. We subsequently fit primitives clusters extract text detection recognition. final output our proposed an information-rich twin buildings contains information, semantic categories which valuable many aspects, condition monitoring, facility maintenance management. In summary, presents nearly fully-automated pipeline enrich details provides comprehensive case study.